1. Field of the invention
The present invention relates to a video transmission apparatus used in a video teleconference system or the like.
2. Description of the Related Art
FIG. 10 is a system configuration diagram of a video teleconference terminal of the prior art proposed in an article "Video teleconference system" (Mitsubishi Electric Corporation Technical Report, vol.64, No.8, August 1990). In the drawing, the monitor 300 is a motion picture monitor for human figures which displays received motion pictures of teleconference participants at a remote location, or the transmitted motion pictures of participants at the present location, by switching between the two. The motion picture monitor 300 has an aspect ratio (width to height ratio) of 4:3. A microphone 301 is used for the input of the conversational voices of the teleconference participants. Voices signals inputted to the microphone 301 are transmitted under the control of a voice controller 302. A camera 303 is a human image capturing camera having an aspect ratio of 4:3 used in capturing the images of the conference participants. A loudspeaker 304 is used to provide the amplified voices of the conference participants at the remote location, these voices' having been transmitted. A monitor 307 is a still picture monitor used to display still pictures transmitted from the remote location or still pictures captured at the present location.
An operator of the video teleconference terminal uses an operation pad 308 to zoom in or out, to focus and orient the human image capturing camera 303, to select the pictures to be displayed on the monitor 300, to transmit the still pictures and control the volume of the loudspeaker 304 output.
A document image capturing camera 309 captures still pictures of documents or small solid objects and converts the pictures into electrical signals. A video controller 310 carries out control operations such as switching of input pictures from the cameras 303 and the document image capturing camera 309 and switching of video signals from a video teleconference terminal at the remote location, according to the operator's control with the operation pad 308.
The video teleconference terminal is capable of entering handwritten characters and drawings by means of an electronic writing pad 311 and a wireless pen 312. The video teleconference terminal is also capable of entering characters and drawings by means of an electronic white board 313. A white board monitor 314 displays the characters and drawings entered via the electronic writing pad 311 or the electronic white board 313. A video printer 315 produces hard copies of the characters and drawings displayed on the white board monitor 314. An auxiliary controller 316 controls the communication with the writing pad and the electronic white board of the video teleconference terminal at the remote location.
A video transmitter 317 has a video CODEC and an audio codec, and transmits signals from a voice controller 302, a video controller 310 and the auxiliary controller 31 to the remote location. A digital network 318 is an ISDN network (integrated services digital network) such as INS64 (information network system 64) or INS1500 which connects the video teleconference terminal of the present location and the video teleconference terminal at the remote location.
The video teleconference terminal is further provided with a digital facsimile 319 which transmits and receives text messages, an ordinary telephone 320 and a data terminal 321 for data file transmission. A time-division multiplexer (TDM) 322 transmits data of the digital facsimile 319, the telephone 320 and the data terminal 321 after time-division multiplexing of the data. A high-speed digital leased line 323 permanently connects the video teleconference terminals at the present and remote locations.
In an actual session of video teleconference, the video teleconference terminals shown in FIG. 10 as described above are installed at the present location and the remote location (location of the other party in the conference). Communication between these video teleconference terminals constitutes the video teleconferencing.
Now the operation of the video teleconference terminal of the prior art shown in FIG. 10 will be described below.
First the operations at the transmitting terminal will be described.
Voices of participants are received by a microphone 301. The received voice signals are input to the voice controller 302 where feedback from the loudspeaker 304 is removed by an echo canceler, before being sent to the video transmitter 317. Pictures of the participants captured by the human image capturing camera 303 are transmitted to the video transmitter 317 as motion pictures to be transmitted, via the video controller 310. Still pictures of documents and small solid objects captured by the document image capturing camera 309 are also transmitted to the video transmitter 317 via the video controller 310.
Characters and graphics entered in handwriting by the participants of the conference on the writing pad 311 or the electronic white board 313 are transmitted via the auxiliary controller 316 to the video transmitter 317. The video transmitter 317 transmits the audio, video, text and graphic data received from the voice controller 302, the video controller 310, and the auxiliary controller 316 to the video teleconference terminal at the remote location via an ISDN link 318, or by using a high-speed digital leased line 323 via the TDM 322.
Now the operations at the receiving end will be described below.
The audio, video, text and graphics data received by the video transmitter 317 at the receiving end, from the video teleconference terminal at the remote location is appropriately divided among the audio controller 302, the video controller 310 and the auxiliary controller 316. The audio signals are processed in the audio controller 302 and output from the loudspeaker 304, while motion picture signals are processed by the video controller 310 and displayed on the human picture monitor 300. Still picture signals are processed by the video controller 310 and displayed on the still picture monitor 307, while the text data and graphics data are processed by the auxiliary controller 316 and displayed on the white board monitor 314 or printed out on the video printer 315. Signals from the digital facsimile 319, the telephone 320 and the data terminal 321 of the video teleconference terminal at the other location are transmitted via the TDM 322 to a terminal of the same kind, where they are output.
Now, a portion of this apparatus related to the video signal transmission will be described in detail below with reference to FIG. 11. In FIG. 11, components identical with those shown in FIG. 10 will be identified with the same numerals.
In FIG. 11, a monitor 300 is a motion picture monitor having an aspect ratio of 4:3. A camera 303a is a panoramic camera generating a video signal 333 having an aspect ratio of 4:3. A motor-driven camera 303b is capable of varying the camera orientation and the magnification factor of the zooming operation by means of a motor, via remote control. It is also capable of focusing at a near field or far field by means of a motor, also via remote control. The motor-driven camera 303b generates a video signal 334 having an aspect ratio of 4:3.
The video signal transmitter 317 carries out digital compression and encoding of the video signal 332 to be transmitted and multiplexes it with other signals, before sending it over the ISDN network 318. The video signal transmitter 317 demultiplexes a signal received from the ISDN network 318 and carries out digital decoding and expansion of the compressed coded digital video signal, thereby generating the received video signal 331. A transmission video signal selector 324 is a switch to select which, of the video signal 333 from the camera 303a or the video signal 334 from the motor-driven camera 303b, should be output, and is controlled by means of a transmission video signal selection interface(I/F) 354 of the video controller 310. The video signal selected by the selector 324 is output to the video signal transmitter 317 and to a monitoring video signal selector 325. The monitoring video signal selector 325 is a device to select video signals to be displayed on the monitor 300 of the video teleconference terminal at the present location, and selects one from the received video signal 331, which is output from the video signal transmitter 317, and the transmission video signal 332 of the video teleconference terminal at the present location, which is output from the transmission video signal selector 324, and then outputs it as the monitoring video signal. The monitoring video signal selector 325 is controlled by means of a monitoring video selection interface 355 of the video controller 310.
The video controller 310 controls the motor-driven camera 303b, the transmission video signal selector 324 and the monitoring video signal selector 325. The video controller 310 comprises a CPU 350, a memory 351, a bus 352, a camera control interface 353, a transmission video signal selection interface 354, the monitoring video signal selection interface 355 and an operation pad interface 356. An operator or user can control the video controller 310 to switch the video signals by operating keys of the operation pad 308.
The CPU 350 carries out operations required to control the interfaces. The memory 351 comprises a program storing ROM, a working data storing RAM, and other components. The CPU 350, the memory 351 and input/output interfaces are connected by the bus 352. The camera control interface 353 is used in the control of the motor-driven camera 303b. The transmission video signal selection interface 354 controls the transmission video signal selector 324. The monitoring video signal selection interface 355 controls the monitoring video selector 325. The operation pad input interface 358 is an interface for reading, the key operation status of the operation pad 308.
FIG. 12 schematically shows the overview of the operation pad 308. In FIG. 12, a button 381 is used to select the panoramic camera 303a, and a button 382 is used to select the motor-driven camera 303b. A button 383 is used to display the transmission video signal 332 on the monitor 300, and a button 384 is used to display the received video signal 331 on the monitor 300. Buttons 365 through 368 and buttons 369 through 372 are used to control the motor-driven camera 303b. The button 365 is used to zoom in and the button 366 is used to zoom out. The buttons 367 and 368 are used for focusing. Namely the button 367 is used to focus at far points and the button 368 is used to focus at near points. The buttons 369 through 372 are used to orient the motor-driven camera 303b, specifically the button 369 is to direct the camera 303b upward, the button 370 is to direct the camera downward, the button 371 is to direct the camera to the left and the button 372 is to direct the camera to the right. The video controller 310 switches the video signals according to such key operations of the operation pad 308 as have been described above.
The buttons 373 through 376 are used to preset the operations. A series of button operations can be registered corresponding to one of these preset buttons, so that pressing the preset button has the effect of causing the video controller to carry out the same series of operations as have been registered. To zoom in on a certain participant during a session of video teleconference, for example, a series of operations including directing the camera, zooming up and focusing are required. In the case where a particular participant should frequently be zoomed in on, the series of operations required can be registered as a preset button operation, which makes it possible to zoom in on the particular participant simply by pressing the preset button. Pressing one button is sufficient to set the camera angle. The operation pad shown in FIG. 12 is provided with four buttons by way of example. The button 377 is a registration button, used to register the operation information of the camera in correspondence to the preset buttons 373 through 376. This registration is carried out by storing the operation information of each preset button in the video controller 310.
Now the operations for video signal transmission with the conventional video teleconference terminal will be described below with reference to FIG. 11 and FIG. 12. When the power is turned on, the video controller 310 shown in FIG. 11 is initialized. That is, the video controller 310 controls the motor-driven camera 303b to be directed to the initial camera angle which has been preset. The video controller 310 also controls the transmission video signal selector 324 by means of the transmission video signal selection interface 354 to thereby select the video signal 334 output from the motor-driven camera 303b, and controls the monitoring video selector 325 by means of the monitoring video selection interface 355 to thereby select the transmission video signal 332. Thereafter the video controller 310 monitors the key operations on the operation pad 308 by means of the operation pad input interface 356 and, when a key is pressed, carries out the control operation corresponding to the key.
When a panoramic camera button 361 of the operation pad 308 is pressed, the operation pad input interface 356 of the video controller 310 recognizes the operation of the panoramic camera button 361. Then the transmission video signal selection interface 354 directs the transmission video signal selector 324 to select the video signal 333 output from the panoramic camera 303a. When a motor-driven camera button 362 of the operation pad 308 is pressed, the operation pad input interface 356 recognizes the operation. Then the transmission video signal selection interface 354 directs the transmission video signal selector 324 to select the video signal 334 output from the motor-driven camera 303b. The selected video signal is handled as the transmission video signal 332. The video signal transmitter 317 multiplexes the transmission video signal 332 with other signals and sends the resultant signal over the ISDN network 318.
When the "transmitting picture" button 363 of the operation pad 308 is pressed, the operation pad input interface 356 recognizes the operation. This is followed by the CPU 350 directing the monitoring video signal selection interface 355 to switch the monitoring video signal selector 325 so that it outputs the video signal 332 to the monitor 300. This means that, when an operator presses the transmitting picture button on the operation pad, the picture captured by the panoramic camera or by the motor-driven camera located at the present location is displayed on the monitor 300.
When the "received picture" button 384 of the operation pad 308 is pressed, on the other hand, the operation pad input interface 356 recognizes the operation. Then under the direction of the CPU 350, the monitoring video signal selection interface 355 directs the monitored video signal selector 325 to output the video signal 331 which is received from the video signal transmitter 317 to the monitor 300. This means that, when an operator presses the "received picture" button, the video signal received from the remote terminal (remote location) is displayed on the monitor 300.
When the "zoom-in" button 365 or the "zoom-out" button 366 of the operation pad 308 are pressed, the camera control interface 353 directs the motor-driven camera 303b to zoom in or zoom out, respectively. When the "far" button 387 or the "near" button 388 of the operation pad 308 are pressed, the camera control interface 353 directs the motor-driven camera 303b to focus at a distant point or at a near point, respectively. When the "up", "down", "left" or "right" button 369.about.372 of the operation pad 308 are pressed, the camera control interface 353 directs the motor-driven camera 303b to the specified direction.
When one of the preset buttons 373 through 376 (P1 through P4), is pressed, the operation pad input interface 356 recognizes the operation and causes the camera control interface 353 to control the motor-driven camera 303b according to the camera view information (operation information: focal length of the zoom lens, direction on focusing (far, near) and camera orientation) stored in the memory 351 in correspondence to the preset buttons. In the case where the registration button 377 is pressed, immediately followed by the pressing of one of the preset buttons P1 through P4, the operation pad input interface 356 recognizes the operation so that the CPU 350 stores the information such as camera angle and zooming which are set at that time in the memory 351 as the camera view information in correspondence to the preset button which is pressed.
In order to conduct a video teleconference session by using such a video transmission apparatus, video teleconference terminals located at the present and remote locations provided with the functions of the transmitter station and receiver station are connected so that each party can view the image of another party. The operation of transmitting video signals from one terminal to the other terminal will be described below in detail.
To transmit video signals from the transmitting station to the receiving station, an operator at the transmitting station presses the "panoramic camera" button 361 or the "motor-driven camera" button 362 of the operation pad 303 while an operator at the receiving station presses the "received picture" button 364 on the operation pad 303. This causes the transmission video signal selector 324 of the transmitting station to select the video signal from either the panoramic camera 303a or the motor-driven camera 303b according to the instruction from the video controller 310, and to output the signal as the transmission video signal 332 to the video signal transmitter 317. The video signal transmitter 317 of the transmitting station multiplexes the transmission video signal 332 with other signals and sends the resultant signal over the digital network 318.
The video signal transmitter 317 of the receiving station which has received the signal from the transmitting station via the network 318 demultiplexes the received signal and arranges it as the received video signal 331, then outputs the received video signal 331 to the monitoring video signal selector 325. Then, because the "received picture" button 364 of the operation pad 308 is pressed, the monitoring video signal selector 325 outputs the received video signal 331 to the monitor 300. As a result, the monitor 300 of the receiving station displays the images sent by the transmitting station.
In such a conventional video teleconference terminal, the panoramic camera 303a, the motor-driven camera 303b and the monitor 300 are all designed to handle pictures having an aspect ratio of 4:3. Meanwhile, recently there has been a strong demand for the use of a wide picture format having an aspect ratio of 16:9, as well as the ordinary picture format of 4:3aspect ratio, in the field of video teleconferencing.
However, when cameras and monitors having different aspect ratios are used, such as a standard camera having an aspect ratio of 4:3 for the motor-driven camera 303b, a wide camera having an aspect ratio of 16:9 for the panoramic camera 303a and a wide monitor having an aspect ratio of 16:9 for the monitor 300, for example, in the video teleconference terminal, there have been such problems as described below. These problems will be described with reference to FIG. 13A and FIG. 13B.
In FIG. 13A and FIG. 13B, the panoramic camera 303a is a wide camera having an aspect ratio of 16:9, and the motor-driven camera 303b is a standard camera having an aspect ratio of 4:3. The monitor which displays the pictures at the receiving terminal is a wide monitor having an aspect ratio of 16:9.
Usually it is impossible to send video signals of different aspect ratios over the same communication line. For this reason, the video signal transmitted over the communication line is made to have the aspect ratio of 4:3 in FIG. 13A and FIG. 13B. Therefore pictures captured by the wide camera having the aspect ratio of 16:9 are compressed in the horizontal direction and converted to a video signal having the aspect ratio of 4:3.
FIG. 13A shows a case where video signals having an aspect ratio of 4:3 are displayed on the wide monitor 100 as pictures having the aspect ratio of 4:3. FIG. 13B shows a case where video signals having the aspect ratio of 4:3 are displayed on the wide monitor 100 after being expanded horizontally to the aspect ratio of 16:9.
In FIG. 13A, the motor-driven camera 303b captures the image in a field of view 400 with the aspect ratio of 4:3 to generate a video signal 401 having the aspect ratio of 4:3. The wide camera 303a compresses a field of view 403 having an aspect ratio of 16:9 to generate a video signal 404 having an aspect ratio of 4:3. The wide monitor 100 displays the pictures while switching between the video signals from the wide camera 303a and the motor-driven camera 303b.
As shown in FIG. 13A, when the video signal having an aspect ratio of 4:3 is displayed on the wide monitor 100 as a picture having an aspect ratio of 4:3, the picture 402 from the motor-driven camera 303b is displayed without distortion, although a picture 405 from the wide camera 303a suffers from distortion of vertical elongation.
On the other hand, when the input video signal is displayed on the wide monitor 100 after being expanded horizontally to an aspect ratio of 16:9 as shown in FIG. 13B, a picture 415 from the wide camera 303a is displayed without distortion although the picture 412 from the motor-driven camera 303b suffers from distortion of horizontal elongation.
The conventional video teleconference terminal has a drawback in that, when pictures of different aspect ratios are transmitted together, images of one aspect ratio are displayed with distortion on the receiving terminal and cannot be reproduced correctly, as described above.
Also in such applications as video teleconferencing, when a person is zoomed in or using a standard camera, for example, it is better to display the picture with the aspect ratio of 4:3 even on a wide monitor, as shown in FIG. 14A, because an image of one person can be better recognized with the aspect ratio of 4:3. When a picture of three or four persons are displayed as shown in FIG. 14B, however, human figures become relatively small in a picture frame having an aspect ratio of 4:3 thereby leaving relatively large unnecessary areas at the top and bottom of the screen, thus canceling the advantage of using the wide monitor. When a picture of a single person is displayed with an aspect ratio of 4:3 and an image of three or four persons is displayed over the full screen of the wide monitor having an aspect ratio of 16:9, on the other hand, troublesome operations of switching between different aspect ratios are required.